Customized presentation of data

ABSTRACT

Provided herein are various systems and methods for monitoring how users interact with medical imaging exams to automatically determine the view order and importance of various series within medical imaging exams as a function of a particular user, exam type, clinical information, and/or other characteristic of medical data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 13/495,991, filed Jun. 13, 2012, which claims the benefit ofpriority under 35 U.S.C. § 119(e) of U.S. Provisional Application No.61/496,973, filed Jun. 14, 2011, the disclosure of both prior-filedapplications are hereby incorporated by reference in their entirety.

BACKGROUND

There is a need for innovations that increase the efficiency andaccuracy of interpretation of medical imaging exams.

SUMMARY

Provided herein are various systems and methods for monitoring how usersinteract with medical imaging exams to automatically determine the vieworder and importance of various series within medical imaging exams as afunction of a particular user, exam type, clinical information, and/orother characteristic of medical data.

In one embodiment, a method of ordering a plurality of image series of amedical exam comprises determining an exam characteristic associatedwith a medical exam, accessing interaction data of a user of a computingdevice, the computing device comprising one or more computer processors,the interaction data storing associations between exam characteristicsand respective orders in which series of images associated withrespective exam characteristics were selected for display by the user,and determining, based on interaction data indicating respective ordersin which series of images associated with the determined examcharacteristic were viewed, a custom ordering of the series of the exam.

In some embodiments, the exam characteristic comprises one or more of anexam type, exam modality, clinical indication and/or other clinicalinformation, medical history of a patient, or risk factors associatedwith the patient. In some embodiments, the computing device isconfigured to display images of the plurality of series in an orderindicated in the custom ordering. In some embodiments, the computingdevice is configured to preload, process with computer aideddiagnostics, and/or generate reconstructions of images of the pluralityof series in an order indicated in the custom ordering. In someembodiments, the method further includes accessing interaction data ofthe user of the computing device, the interaction data storingassociations between exam characteristics and relative importance levelsof respective series associated with exams having respective examcharacteristics, wherein the custom ordering of the series of the examis further based on the importance levels associated with the determinedexam characteristic. In some embodiments, the importance level ofrespective series is based on one or more of a number of images ofrespective series that are added to a montage, a number of images of therespective series that are marked as key images, an order in whichrespective series are selected for display, a frequency that images ofrespective series are used for measurements, or a frequency that imagesfor respective series are selected for inclusion in a report. In someembodiments, the interaction data further includes interaction data ofother users.

In one embodiment, a method comprises determining one or morecharacteristics of an exam to be used in determining a custom orderingof respective series of images of the exam, identifying interaction dataassociated with the determined one or more characteristics of the exam,the interaction data indicating interactions of one or more users withimages of respective image series of other exams having the determinedone or more characteristics, and determining, based on the identifiedinteraction data, a custom ordering of series of the exam.

In some embodiments, the one or more characteristics are determinedbased on user preferences, group preferences, site preferences, systempreferences, and/or default software preferences. In some embodiments,the one or more characteristics comprise one or more of an exam type,exam modality, clinical indication and/or other clinical information,medical history of a patient, or risk factors associated with thepatient. In some embodiments, the one or more characteristics compriseonly a type of the exam. In some embodiments, the one or morecharacteristics comprise only clinical indication of the exam. In someembodiments, the one or more users comprise only the user. In someembodiments, the one or more users comprise one or more other users. Insome embodiments, the one or more other users comprise users associatedwith a same group as the user, users associated with a same specialty asthe user, and/or users designated as experts with reference to examshaving the determined one or more characteristics. In some embodiments,the method further includes determining the interaction data based on anorder in which the one or more users selected for display respectiveimage series of other exams having the determined one or morecharacteristics. In some embodiments, the method further includesdetermining the interaction data based on data indicating which imagesseries of other exams having the determined one or more characteristicsinclude images that are marked as key images or selected for inclusionin a montage. In some embodiments, the data comprises importance scoresfor respective series associated with exams having the determined one ormore characteristics. In some embodiments, importance scores forrespective image series are weighted based on a quantity of images ofrespective image series that are marked as key images or selected forinclusion in a montage. In some embodiments, the method further includesusing the determined custom ordering as an order of displaying series ofthe exam, preloading series of the exam, processing series of the examwith computer aided diagnostics, and/or generating reconstructions ofimages of series of the exam. In some embodiments, use of the determinedcustom ordering is determined based on user preferences, grouppreferences, site preferences, system preferences, and/or defaultsoftware preferences.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system diagram which shows various components of a systemconfigured for displaying information utilizing certain systems andmethods described herein.

FIG. 2 is a system diagram which shows various components of a systemfor managing data (e.g., medical or non-medical data) utilizing certainsystems and methods described herein.

FIG. 3 illustrates example computing devices that may be used to performvarious processes discussed herein.

FIG. 4a illustrates example arrangements of image series of an exam, inparticular, a brain MRI in the example of FIG. 4 a.

FIG. 4b is a flowchart illustrating one embodiment of a method ofmonitoring user behavior to collect interaction data, such as seriesviews order.

FIG. 5a is a table illustrating example series importance data that maybe derived based on user interaction data (of a single or multipleusers) in order to determine series importance as related to respectiveclinical indications.

FIG. 5b is a flowchart illustrating one embodiment of a method formonitoring user behavior to collect information related to seriesimportance.

FIG. 6 illustrates exemplary orderings of series based on seriesimportance for the respective exam indication.

FIG. 7 illustrates a computing device as images are selected anddisplayed on a computing device.

FIG. 8 illustrates a computing device as images are selected anddisplayed on the computing device.

FIG. 9 illustrates additional views of the image series discussed withreference to FIGS. 7 and 8, again with the series ordered according toseries view order and/or series importance, as discussed with referenceto FIG. 8.

FIG. 10 illustrates an embodiment where the user may change series andimages using touch gestures.

FIG. 11 is a flowchart illustrating one embodiment of a method ofpre-loading series based on a custom series order, such as based onseries view order of the user and/or series importance for variousseries.

FIG. 12 is a flowchart illustrating one embodiment of the method ofpresenting series of an exam based on determined series importance.

FIG. 13 illustrates an example screen from a computing device configuredto display images from multiple image series concurrently.

FIG. 14 illustrates an arrangement of various image series, wherein sixof the image series may be displayed on a display device (or multipledisplay devices) concurrently.

FIG. 15 illustrates a tablet device displaying the first six imageseries of the exam discussed with reference to FIG. 14.

DETAILED DESCRIPTION

Embodiments of the disclosure will now be described with reference tothe accompanying figures, wherein like numerals refer to like elementsthroughout. The terminology used in the description presented herein isnot intended to be interpreted in any limited or restrictive manner,simply because it is being utilized in conjunction with a detaileddescription of certain specific embodiments of the disclosure.Furthermore, embodiments of the disclosure may include several novelfeatures, no single one of which is solely responsible for its desirableattributes or which is essential to practicing the embodiments of thedisclosure herein described.

As used herein, the terms “viewer” and “user” are used interchangeablyto describe an individual (or group of individuals) that interfaces witha computing device. Users may include, for example, doctors,radiologists, hospital staff, or other individuals involved inacquisition, analysis, storage, management, or other tasks related tomedical images. In other embodiments, users may include any individualsor groups of individuals that generate, transmit, view, and/or otherwisework with images of any type. Any discussion herein of user preferencesshould be construed to also, or alternatively, include user grouppreferences, site preferences, system preferences, and/or defaultsoftware preferences.

Depending on the embodiment, the methods described with reference to theflowcharts, as well as any other methods discussed herein, may includefewer or additional blocks and/or the blocks may be performed in adifferent order than is illustrated. Software code configured forexecution on a computing device in order to perform the methods may beprovided on a tangible computer readable medium, such as a compact disc,digital video disc, flash drive, hard drive, memory device or any othertangible medium. Such software code may be stored, partially or fully,on a memory of a computing device (e.g., RAM, ROM, etc.), such as thecomputing system 150 (see discussion of FIG. 1, below), and/or othercomputing devices illustrated in the figures, in order to perform therespective methods. For ease of explanation, the methods will bedescribed herein as performed by the computing system 150, but themethods are not limited to performance by the computing system 150 andshould be interpreted to include performance by any one or more of thecomputing devices noted herein and/or any other suitable computingdevice.

Definitions

In order to facilitate an understanding of the systems and methodsdiscussed herein, a number of terms are defined below. The terms definedbelow, as well as other terms used herein, should be construed toinclude the provided definitions, the ordinary and customary meaning ofthe terms, and/or any other implied meaning for the respective terms.Thus, the definitions below do not limit the meaning of these terms, butonly provide exemplary definitions.

Medical imaging exam: Medical imaging exams comprise data related to amedical procedure, such as medical images, medical reports, and/orrelated information. Medical imaging exams can be acquired by a numberof different medical imaging techniques, including computed tomography(CT), magnetic resonance imaging (MRI), ultrasound, nuclear medicine,positron emission computed tomography (PET), digital angiography,mammography, computed radiography, digital radiography, fluoroscopy,images generated in medical pathology and endoscopy, and any otherimaging techniques. Medical imaging exams may also include text reports,graphs, numerical information such as measurements, movies, sounds orvoice data, and/or any other information that may be stored in digitalformat. Although much of the discussion herein is with reference tomedical imaging exams, the systems and methods described may be usedwith other types of images and data. Thus, any reference to medicalimages may alternatively be construed to cover any other type of image.

Series: Medical imaging exams are typically organized into one or moreseries, with each series including one or more images. Images in aseries typically share one or more common characteristic, for examplethe type of anatomic plane and/or image orientation. Series may becharacterized by their type. For example, series may be acquired usingdifferent pulse sequences, acquired in different anatomic planes, andacquired before or after administration of intravenous contrastmaterial. In some embodiments a series may include other types ofinformation, such as text reports, graphs, numerical information such asmeasurements, movies, sounds or voice data, and/or any other informationthat may be stored in digital format.

Hanging Protocol: A hanging protocol indicates, and may be used todetermine, a layout of series on one or more computer displays. Forexample, a user may prefer the arrangement shown in view 1310 of FIG. 13for display of brain MRI exams, where the Sagittal T1 series isdisplayed in the top-left frame, the Axial T1 series is displayed in thetop-middle frame, etc. The user may prefer different arrangements on thebasis of a variety of factors, e.g., exam type, clinical indication,computer hardware, etc.

Interaction Data: Interaction data is information indicating how a userinteracts with medical images. For example, interaction data mayindicate an order in which a particular viewer views images from variousseries types. Interaction data may indicate how long a user interactswith particular images, image series, or other pieces of medical data.Interaction data may indicate operations a user performs on particularimages, image series, or other pieces of medical data. Interaction datamay be user specific (e.g., each user can have a set of interactiondata). Interaction data may be associated with a group of users (e.g., aradiology group may have group interaction data, possibly in addition touser specific interaction data, or interaction data may be associatedwith a subgroup, such as radiologists with expertise in a particulararea of radiology). Non-experts may utilize interaction data collectedfrom experts in a field. Thus, a user's interaction data may includeinteraction data of a particular user and/or interaction data of a groupof users. Interaction data may be obtained in various manners, such asby monitoring data that is built into image viewing software (e.g., PACSsoftware) or third-party software that interacts with image viewingsoftware. Interaction data may be stored in any format and madeavailable to software modules that determine adjustments to viewingpreferences of a user based on the particular user's (and/or groups towhich the user belongs) interaction data.

Series View Order: Series view order is an order in which series of anexam are viewed, such as by a particular user. Irrespective of the wayvarious series are displayed on computing device, as determined by ahanging protocol, for example, a user may view the series in an orderthat is determined by the particular user's thought process and/orroutine, for example. Thus, different users may have different seriesview orders, even for the same exam type using the same hangingprotocol. In addition, a user's routine for viewing the various seriesmay vary depending on the clinical indication for performing the exam,viewing location, viewing device, and/or other information related tothe user or patient.

For example, for a brain MRI performed for “Possible MultipleSclerosis”, a user may prefer to first view the Sagittal FLAIR series,followed by the Axial FLAIR series, etc. For a different clinicalindication, though, such as “Acute stroke”, the same user may routinelyview the Axial Diffusion series first as it is the most sensitive seriesfor detection of acute infarction, followed by the Axial FLAIR series,etc. Thus, even if the hanging protocol for the two medical imagingexams is identical, the user may view images of the medical imagingexams in a different order. Thus, series view order may vary amongusers, change over time for a user, and/or vary for a user based on theclinical information associated with the exam and/or other informationregarding the user, the viewing environment, and/or the exam.

In various embodiments described herein, a series view order may be:

-   -   Determined automatically by a computing device based on the        user's interaction data (and/or interaction data of one or more        groups). A series view order may be determined upon request of        an exam from a user (e.g., in real-time using current        interaction data of the user) and/or may be determined based on        concurrent or earlier analysis of the user's interaction data or        a groups interaction data. For example, a user's series view        order may be re-determined monthly, at the user's request, or in        response to one or more predefined user or system actions.    -   Set explicitly or determined automatically for a user, user        group, site, etc.    -   Automatically correlated with clinical information (or other        characteristic of an exam) so that a different series view order        may be associated with different clinical information (or other        characteristic of an exam).    -   Used by the computing device to prioritize various operations        related to an exam, which may optimize a user's access and/or        review of the medical data. For example, if the user accesses a        brain MRI with the clinical information “Acute Stroke” and his        series view order indicates that the Axial Diffusion series is        the first series in his typical series view order for that        indication (e.g., based on the user's interaction data), the        Axial Diffusion series may be communicated to the user's        computing device first. This may increase the speed that the        user can access that series, increasing the efficiency of the        user. In another embodiment, an exam type may be used to        determine the order in which series are transmitted, processed,        and/or displayed to the user, regardless of the associated        clinical indication (or other clinical information) associated        with the exam. Thus, series view order may be associated with        various characteristics of exams, such as exam type, exam        modality, and/or any clinical information associated with the        exam.    -   Used to determine an order of operations related to image        processing that could occur on a client or server, for example,        creation of MPR or 3D volumetric rendered images, processing        with Computer Aided Diagnosis (CAD) software, etc.    -   Used to organize the display of information based on a computing        device type and/or for a particular computing device. For        example, a user may prefer to display only a single image frame        when viewing exams on a smartphone, allowing the display of only        one series at a time.    -   Used to determine the order that the various series are        displayed, e.g., displaying the series in an order indicated by        a series view order.

Series Importance: Series importance is an indication of importance ofrespective series and/or images within a series. Series importance maybe determined based on several factors, discussed below, such asindications of importance of images in respective series that areprovided by a particular user and/or other user's that have previouslyviewed series having similar characteristics (e.g. same series type andclinical indication). In some embodiments, interaction data of aparticular user is monitored to determine the relative importance ofvarious series and to determine the series importance for the user.

In medical imaging exams, various series may vary in their sensitivityand specificity for detecting various abnormalities. Once a radiologisthas viewed a medical imaging exam, it may be useful to other doctors whomay later view the exam (or other similar exams) to be provided asummary of the important findings of the exam in the form of a fewselected images. Computing systems used by radiologists to view medicalimaging exams may allow users to designate certain images within an examas “key images.” By tracking the frequency that images are chosen by theradiologist from various types of series, the various series may beranked in terms of “series importance”.

In various embodiments, the series importance may be based on thefrequency that images within series are viewed, chosen as key images,chosen for inclusion in a montage of selected images that the userchooses to summarize the exam, chosen for inclusion in a report, and/orused for measurements. In other embodiments, additional and/or differentinteraction data may be used to determine series importance.

Series Importance may vary from user to user and for a single user mayvary based on a variety of actions, such as clinical informationassociated with a patient or a patient's medical exam, for example. Invarious embodiments described herein, series importance may be:

-   -   Determined automatically by a computing device based on the        user's interaction data (and/or interaction data of one or more        groups in which the user is a member and/or data of one or more        groups in which the user is not a member, e.g. a group of expert        users). A series importance may be determined upon request of an        exam by a user (e.g., in real-time using current interaction        data of the user) and/or may be predetermined based on earlier        analysis of the user's interaction data and/or determined based        on current analysis of prior interaction data. For example,        series importance for respective exam types may be re-determined        monthly or at the user's request.    -   Set explicitly or determined automatically for a user, user        group, site, etc.    -   Automatically correlated with clinical information (or other        characteristic of an exam) so that a different series importance        may be associated with different clinical information (or other        characteristic of an exam).    -   Used by the computing device to prioritize various operations        related to a series or exam (e.g., multiple series), which may        optimize a user's access and/or review of the medical data. For        example, if the user accesses a brain MRI with the clinical        information “Acute Stroke” and the user's series importance        indicates that the Axial Diffusion series is the most        “important” series for that particular clinical indication, the        Axial Diffusion series may be communicated to the computing        device first. This may increase the speed that the user can        access that series, increasing the efficiency of the user.    -   Used to organize the display of information based on a computing        device type and/or for a particular computing device. For        example, a user may prefer to display only a single image frame        when viewing exams on a smartphone, allowing the display of only        one series at a time. The series importance may be used to        organize the order that the various series are displayed, e.g.,        displaying the most “important” series first followed by the        other series in series view order. This may serve as a form of        cognitive augmentation, where the user's attention is directed        to the most “important” series first, e.g., a series that has        been determined to be of most importance for the particular        clinical indication associated with the exam.    -   Aggregated among users or groups of users. For example, the        series importance data from a group of neuroradiologists may be        designated as the “expert series importance” information. This        could then be used by less experienced users to guide the        presentation of information for that group, a form of cognitive        augmentation.

INTRODUCTION

As discussed further herein, interaction data of a user may bemonitored, stored, and/or used in various manners, such as in order todetermine series view order and/or series importance to be used indisplaying an exam series to the user. In various embodimentsinteraction data, as well as data derived from the interaction data,such as series view order and/or series importance, may be used to:

-   -   Automatically organize presentation of exam components (e.g.,        series of an exam) based on a predicted importance of respective        exam components based on interaction data of a user requesting        an exam in combination with various other exam, environment,        and/or other characteristics, such as exam type, clinical        indication, user, user group, or other characteristic of a user        or user viewing environment. This may increase efficiency as        well as serve as a form of cognitive augmentation by        automatically directing the user's attention to the most        important components of an exam, e.g., based on clinical        information of the exam.    -   Enhance reading accuracy and/or evolve exam acquisition        protocols.    -   Increase system responsiveness and physician efficiency by        prioritizing transmission, processing, and/or display of exam        series based on the order the user is likely to need the        information during viewing of an exam based on the user, exam        type, clinical information, etc. This prioritization may be        independent of hanging protocols and may be particularly useful        for web connections, e.g., mobile, cloud based PACS/EMR, etc.

Example Computing System

FIG. 1 is a system diagram which shows the various components of asystem 100 configured for displaying information utilizing certainsystems and methods described herein. As shown, the system 100 mayinclude an information display computing device 150 and may includeother systems, including those shown in FIG. 1.

The information display computing device 150, also referred to herein as“computing device 150” or “device 150,” may take various forms. In oneembodiment, the information display computing device 150 may be acomputer workstation having information display software modules 151. Inother embodiments, software modules 151 may reside on another computingdevice, such as a web server or other server, and the user directlyinteracts with a second computing device that is connected to the webserver via a computer network. The software modules 151 will bedescribed in detail below.

In one embodiment, the information display computing device 150comprises a server, a desktop computer, a workstation, a laptopcomputer, a mobile computer, a smartphone, a tablet computer, a cellphone, a personal digital assistant, a gaming system, a kiosk, an audioplayer, any other device that utilizes a graphical user interface,including office equipment, automobiles, airplane cockpits, householdappliances, automated teller machines, self-service checkouts at stores,information and other kiosks, ticketing kiosks, vending machines,industrial equipment, and/or a television, for example.

The information display computing device 150 may run an off-the-shelfoperating system 154 such as a Windows, Linux, MacOS, Android, or iOS.The information display computing device 150 may also run a morespecialized operating system which may be designed for the specifictasks performed by the computing device 150.

The information display computing device 150 may include one or morecomputing processors 152. The computer processors 152 may includecentral processing units (CPUs), and may further include dedicatedprocessors such as graphics processor chips, or other specializedprocessors. The processors generally are used to execute computerinstructions based on the information display software modules 151 tocause the computing device to perform operations as specified by themodules 151. The modules 151 may include, by way of example, components,such as software components, object-oriented software components, classcomponents and task components, processes, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,microcode, circuitry, data, databases, data structures, tables, arrays,and variables. For example, modules may include software code written ina programming language, such as, for example, Java, JavaScript,ActionScript, Visual Basic, HTML, Lua, C, C++, or C#. While “modules”are generally discussed herein with reference to software, any modulesmay alternatively be represented in hardware or firmware. Generally, themodules described herein refer to logical modules that may be combinedwith other modules or divided into sub-modules despite their physicalorganization or storage.

The information display computing device 150 may also include memory153. The memory 153 may include volatile data storage such as RAM orSDRAM. The memory 153 may also include more permanent forms of storagesuch as a hard disk drive, a flash disk, flash memory, a solid statedrive, or some other type of non-volatile storage.

The information display computing device 150 may also include or beinterfaced to one or more display devices 155 that provide informationto the users. Display devices 155 may include a video display, such asone or more high-resolution computer monitors, or a display deviceintegrated into or attached to a laptop computer, handheld computer,smartphone, computer tablet device, or medical scanner. In otherembodiments, the display device 155 may include an LCD, OLED, or otherthin screen display surface, a monitor, television, projector, a displayintegrated into wearable glasses, or any other device that visuallydepicts user interfaces and data to viewers.

The information display computing device 150 may also include or beinterfaced to one or more input devices 156 which receive input fromusers, such as a keyboard, trackball, mouse, 3D mouse, drawing tablet,joystick, game controller, touch screen (e.g., capacitive or resistivetouch screen), touchpad, accelerometer, video camera and/or microphone.

The information display computing device 150 may also include one ormore interfaces 157 which allow information exchange between informationdisplay computing device 150 and other computers and input/outputdevices using systems such as Ethernet, Wi-Fi, Bluetooth, as well asother wired and wireless data communications techniques.

The modules of the information display computing device 150 may beconnected using a standard based bus system. In different embodiments,the standard based bus system could be Peripheral Component Interconnect(“PCI”), PCI Express, Accelerated Graphics Port (“AGP”), Micro channel,Small Computer System Interface (“SCSI”), Industrial StandardArchitecture (“ISA”) and Extended ISA (“EISA”) architectures, forexample. In addition, the functionality provided for in the componentsand modules of information display computing device 150 may be combinedinto fewer components and modules or further separated into additionalcomponents and modules.

The information display computing device 150 may communicate and/orinterface with other systems and/or devices. In one or more embodiments,the computer device 150 may be connected to a computer network 190. Thecomputer network 190 may take various forms. It may be a wired networkor a wireless network, or it may be some combination of both. Thecomputer network 190 may be a single computer network, or it may be acombination or collection of different networks and network protocols.For example, the computer network 190 may include one or more local areanetworks (LAN), wide area networks (WAN), personal area networks (PAN),cellular or data networks, and/or the Internet.

Various devices and subsystems may be connected to the network 190. Forexample, one or more medical scanners may be connected, such as MRIscanners 120. The MRI scanner 120 may be used to acquire MRI images frompatients, and may share the acquired images with other devices on thenetwork 190. The network 190 may also include one or more CT scanners122. The CT scanners 122 may also be used to acquire images and, likethe MRI scanner 120, may then store those images and/or share thoseimages with other devices via the network 190. Any other scanner ordevice capable of inputting or generating information that can bepresented to the user as images, graphics, text or sound could beincluded, including ultrasound, angiography, nuclear medicine,radiography, endoscopy, pathology, dermatology, etc.

Also connected to the network 190 may be a Picture Archiving andCommunications System (PACS) 136 and PACS workstation 138.

Also connected to the network 190 may be a User Profile Data 160. Theuser profile data 160 may include a database or other data structurethat stores information such as interaction data, series view order,series importance, and/or other data associated with various users. Invarious embodiments, the user profile data 160 may reside within PACSSystem 136, reside within a server accessible on a LAN that isaccessible to the information display computing device 150, and/orreside within a server that is located remote to the information displaycomputing device 150 and accessible via the Internet. In otherembodiments, user profile data 160 may reside locally, withininformation display computing device 150. Information may be stored inthe user profile data 160 (and/or elsewhere) in any computer readableformat such as a database, flat file, table, or XML file, and may bestored on any computer readable medium, such as volatile or non-volatilememory, compact disc, digital video disc, flash drive, or any othertangible medium.

The PACS System 136 is typically used for the storage, retrieval,distribution and presentation of images (such as those created and/orgenerated by the MRI scanner 120 and CT Scanner 122). The medical imagesmay be stored in an independent format, an open source format, or someother proprietary format. The most common format for image storage inthe PACS system is the Digital Imaging and Communications in Medicine(DICOM) format. The stored images may be transmitted digitally via thePACS system, often reducing or eliminating the need for manuallycreating, filing, or transporting film jackets.

The network 190 may also be connected to a Radiology Information System(RIS) 140. The radiology information system 140 is typically acomputerized data storage system that is used by radiology departmentsto store, manipulate and distribute patient radiological information.

Also attached to the network 190 may be an Electronic Medical Record(EMR) system 142. The EMR system 142 may be configured to store and makeaccessible to a plurality of medical practitioners computerized medicalrecords. Also attached to the network 190 may be a LaboratoryInformation System 144. Laboratory Information System 144 is typically asoftware system which stores information created or generated byclinical laboratories. Also attached to the network 190 may be a DigitalPathology System 146 used to digitally manage and store informationrelated to medical pathology.

Also attached to the network 190 may be a Computer Aided DiagnosisSystem (CAD) 148 used to analyze images. In one embodiment, the CAD 148functionality may reside in a computing device separate from informationdisplay computing device 150 while in another embodiment the CAD 148functionality may reside within information display computing device150.

Also attached to the network 190 may be a 3D Processing System 149 usedto perform computations on imaging information to create new views ofthe information, e.g., 3D volumetric display, Multiplanar Reconstruction(MPR) and Maximum Intensity Projection reconstruction (MIP). In oneembodiment, the 3D Processing functionality may reside in a computingdevice separate from information display computing device 150 while inanother embodiment the 3D Processing functionality may reside withininformation display computing device 150

In other embodiments, other computing devices that store, provide,acquire, and/or otherwise manipulate medical data may also be coupled tothe network 190 and may be in communication with one or more of thedevices illustrated in FIG. 1, such as with the information displaycomputing device 150.

As will be discussed in detail below, the information display computingdevice 150 may be configured to interface with various networkedcomputing devices in order to provide efficient and useful review ofmedical examination data that is stored among the various systemspresent in the network. In other embodiments, information displaycomputing device 150 may be used to display non-medical information.

Depending on the embodiment, the other devices illustrated in FIG. 1 mayinclude some or all of the same components discussed above withreference to the Information Display Computer Device 150.

FIG. 2 is a system diagram which shows the various components of asystem 200 for managing data (e.g., medical or non-medical data)utilizing certain systems and methods described herein. As shown, thesystem 200 may include a computing device 250 and may include othersystems, including those shown in FIG. 2.

The computing device 250 may take various forms. In one embodiment, thecomputing device 250 may be a computer workstation having softwaremodules 151. In other embodiments, software modules 151 may reside onanother computing device, such as a web server, and the user directlyinteracts with a second computing device that is connected to the webserver via a computer network. The software modules 151 will bedescribed in detail below.

In one embodiment, the computing device 250 comprises a server, adesktop computer, a workstation, a laptop computer, a mobile computer, aSmartphone, a tablet computer (e.g., the tablet computer 320 of FIG. 3),a cell phone (e.g., the smartphone 330 of FIG. 3), a personal digitalassistant, a gaming system, a kiosk, an audio player, any other devicethat utilizes a graphical user interface, including office equipment,automobiles, airplane cockpits, household appliances, automated tellermachines, self-service checkouts at stores, information and otherkiosks, ticketing kiosks, vending machines, industrial equipment, and/ora television, for example.

The computing device 250 may run an off-the-shelf operating system 154such as a Windows, Linux, MacOS, Android, or iOS. The computing device250 may also run a more specialized operating system which may bedesigned for the specific tasks performed by the computing device 250.

As with computing device 150 described herein with reference to FIG. 1,computing device 250 may include one more computing processors 152, mayinclude memory storage 153, may include or be interfaced to one moredisplay devices 155, may include or be interfaced to one or more inputdevices 156, and may include one or more interfaces 157.

Computing device 250 may communicate and/or interface with other systemsand/or devices via network 190, as described herein with reference toFIG. 1.

Also connected to Network 190 may be a Server 210 that communicates withComputing Device 250, for example allowing communication of images orother data between Server 210 and Computing Device 250.

Example Computing Devices

FIG. 3 illustrates example computing devices that may be used to performvarious processes discussed herein. For example, the computing device150 or 250 could include the smartphone 330 or tablet computer 320 ofFIG. 3. As discussed above, the system and methods described herein maybe implemented on any other suitable computing device, such as thoselisted above.

Example Interaction Data

Radiologists and other physicians may prefer to view series in differentorders based on clinical information. For example, radiologists andother physicians may prefer to first view series that they feel are mostlikely to demonstrate abnormalities for the given clinical indication.For example, in a patient suspected of having had an acute infarct,users may prefer to view the diffusion series first as it is mostsensitive for detection of acute infarcts. Described below are systemsand methods for using interaction data of users in order to optimize anorder of transmitting, processing, and or presenting image series to auser.

FIG. 4a illustrates example arrangements of image series of an exam, inparticular, a brain MRI in the example of FIG. 4A. Arrangement 410illustrates an order in which series of the example brain MRI wereacquired, while arrangement 420 indicates an order in which the variousseries of the brain MRI were actually viewed by a particular user, user1 in this example. Thus, arrangement 410 indicates that the Sagittal T1series was acquired first, followed by Axial T1, Axial FLAIR, etc. Inone embodiment, the order that series are acquired may be arbitrary andmay vary from site to site and scanner to scanner. However, in somecases certain series are acquired in a particular order, e.g., if pre-and post-contrast images are acquired, the pre-contrast scans would beacquired before post-contrast scans.

As shown in arrangement 420, the order in which user 1 actually viewsthe various image series differs from the order in which the series wereacquired (arrangement 410). In particular, arrangement 420 indicatesthat the axial diffusion series was the first viewed series, followed bythe Axial FLAIR, Axial T2, etc. Thus, the series view order, e.g., theway the viewer navigates through the images in a medical imaging exam,may differ among individuals, and may vary among a single user in viewof other factors, such as clinical information associated with theimages and/or factors related to the particular computing device onwhich the user is viewing the medical data.

As will be discussed with regard to different embodiments herein, it isuseful for the device to know the likely order that the user will viewthe series.

FIG. 4a also illustrates a graph 430 that shows example interaction datathat may be collected based on a user's behavior in viewing brain MRIexams associated with the clinical information “Acute Stroke”, such asthe series depicted in arrangements 410 and 420. Graph 430 illustratesan order in which the respective series of the brain MRI exams wereviewed by the user, for example based on interaction data associatedwith the user viewing one or more exams previously. Thus, the graph 430matches the order illustrated in arrangement 420 indicating that theAxial Diffusion series was the first series displayed, the Axial FLAIRseries was second, etc. This viewing order may be used to automaticallydetermine, on average, a user's preferred series viewing order, in thisexample for a Brain MRI performed with the clinical indication of “AcuteStroke.” Accordingly, by collecting this interaction data (e.g., alongwith various other types of interaction data), as users view exams, adatabase of preferred series view order may be acquired, an example ofmachine learning. This data may then be used to predict the preferredseries view order for the user as a function of modality, clinicalindication, and/or other exam or user characteristics, when the userbegins viewing an exam of the same (or similar) modality and/or clinicalindication.

In this example, the clinical indication in the brain MRI is “AcuteStroke,” but a similar or identical series might be obtained for manyother clinical indications.

FIG. 4b is a flowchart illustrating one embodiment of a method ofmonitoring user behavior to collect interaction data, such as seriesviews order.

All flowcharts and/or methods discussed herein may include fewer oradditional blocks and/or the blocks may be performed in a differentorder than is illustrated. Software code configured for execution on acomputing device in order to perform the methods may be provided on acomputer readable medium, such as a compact disc, digital video disc,flash drive, hard drive, memory device or any other tangible medium.Such software code may be stored, partially or fully, on a memory deviceof the computer, such as the computing device 150, computing device 250,and/or any other suitable computing device, in order to perform themethods outlined in the various flowcharts. For ease of explanation, themethods will be described herein as performed by a computing device 150(which refers to either or both of the information display computingdevice 150 or 250); however, the methods may be performed by any othersuitable computing device.

Beginning in block 450, the computing device determines an identity ofthe current user so that the interaction data that is captured can beassociated with the particular user. In some embodiments, information isacquired anonymously or associated with a user group rather than (or inaddition to) an individual user.

In block 452, exam information, such as the modality and seriesinformation, and clinical information associated with the exam and/orpatient may be acquired, such as the clinical indication for the exam.In other embodiments, other clinical information may be utilized, suchas the patient's past medical history, risk factors, etc. In otherembodiments, other information, such as exam type and/or informationfrom prior exams, may be utilized instead of, or in addition, toclinical information. Additionally, information regarding the userand/or the user's viewing environment (e.g., the type of device the useris viewing the images on) may be acquired.

In block 454, interaction data based on the user's behavior as he viewsthe medical imaging exam (e.g., navigates between images of variousseries), is recorded. The interaction data may include the series typeassociated with each image that is displayed, length of time each imageis displayed, user interactions with the image (e.g., resizing, zooming,changing widow levels, cropping, etc.), notations or tags associatedwith the image, previous and/or next images viewed, images and seriesdisplayed from other exams such as prior comparison exams, and/or anyother information associated with the user's interaction with the image.

In block 456, the interaction data is analyzed to determine the seriesview order. In other embodiments, other characteristics of the user'sviewing behavior may be determined based on the interaction data.

In block 458, the interaction data, including the determined orderseries view order, is stored, such as in the user profile data 160 (FIG.1). In one embodiment, a series view order may be generated based on theuser's viewing behavior for all exams of a certain type, such as a brainMRI. In another embodiment, a series view order may be generated forexams of a certain type coupled with clinical information, such as abrain MRI performed to evaluate for possible “acute infarction.” Inanother embodiment, a series view order may be created by combininginformation from multiple users. Thus, a user may have multiple seriesview orders each associated with different combinations of clinicalindications, modalities, display devices, etc.

FIG. 5a is a table illustrating example series importance data that maybe derived based on user interaction data (of a single or multipleusers) in order to determine series importance as related to respectiveclinical indications. In the course of viewing exams, users may interactwith the images to indicate images that are of particular interest. Forexample, a user may choose images to be marked as “key images” or placedin a “montage” of images that communicate the most important findings toother users, such as referring physicians that may later view the exam.

In some cases the images are chosen because they demonstrate anabnormality in the medical imaging exam, for example an enhancing mass.In other cases images are chosen because they show no abnormality, butthe image is chosen from a series that the user feels would be the onemost likely to demonstrate an abnormality if one existed. For example,in a patient imaged for suspected “Acute Stroke”, a radiologist mighttag a normal image from an “Axial Diffusion” series as a key imagebecause that series might be the one expected to be most sensitive fordetection of an acute infarct if one were present.

In the example table of FIG. 5a , the illustrated percentages indicatehow commonly images of the respective series were marked as “key images”(and/or added to a montage) when associated with each of three exampleclinical indications. For example, FIG. 5a indicates that one or moreimage of the axial FLAIR series is marked as a key image 80% of the timewhen the clinical indication is Acute Stroke or Tumor Follow-up.However, one or more images of the axial FLAIR series is marked as a keyimage 95% of the time when the clinical indication is MultipleSclerosis. Thus, while the Axial FLAIR series is important in each ofthe three example clinical indications, that series may be mostimportant in the Multiple Sclerosis clinical indication. In the exampleof FIG. 5A, in the case of Acute Stroke clinical indication, the AxialDiffusion series was the most common series with images marked as keyand/or added to a montage, followed by the Axial Flair series andCoronal GRE series. In the case of Multiple Sclerosis, the Axial Flairseries was the most common series having one or more images marked askey and/or added to a montage, followed by the Sagittal FLAIR series andAxial T1+C series. In the case of Tumor Follow-up, the Axial T1+C serieswas the most common series having one or more images marked as keyand/or added to a montage, followed by the Axial FLAIR series, CoronalT1+C and Sagittal T1+C Series.

Depending on the embodiment, the series importance (e.g. the percentagesillustrated in FIG. 5A) may be determined based on interaction data of asingle user or a group of users. In some embodiments, the user canprovide a preference for which interaction data (e.g., user-specific orgroup) is used in determining series importance for that user, and mayfurthermore indicate a desired weighting of different sources ofinteraction data. For example, a user that is relatively inexperiencedwith viewing images associated with a particular clinical indication maywish to have series importance determined entirely (or primarily) basedon interaction data of other users, for example a group of expert users.However, a user that is very experienced with viewing images associatedwith a particular clinical indication may wish to have series importancedetermined solely (or mostly) based on interaction data of the userhimself.

In other embodiments, rather than a percentage indicator of howfrequently images of respective series are marked as key images and/orplaced in a montage (as discussed above), a scoring algorithm or modelthat considers other factors may be used to rank relative importance ofseries. For example, a scoring model may consider a quantity of imagesof a particular series that are marked as key images and/or placed in amontage in generating an “importance score” for that particular series.Thus, for a particular exam type (or exam type with a particularclinical indication or having other particular clinical information),series of the exam may each have an importance score that is based onthe quantity of images of respective series that are marked as keyimages and/or added to a montage, for example. In some embodiments,different weightings are assigned to image series based on whetherimages were marked as key images or images were placed in a montage. Forexample, a series having one image added to a montage may have adifferent importance score (either higher or lower depending on theparticular scoring algorithm) than a series having one image that wasmarked as a key image.

In some embodiments, importance scores may be determined based oncharacteristics of users from which interaction data was acquired. Forexample, a first user that is an expert in a particular area (e.g., ininterpreting brain MRIs) may have his actions with reference to exams inthat particular area (e.g., his actions in marking images of brain MRIsas key images and/or adding brain MRI images to montages) weighted muchhigher than a user that is relatively inexperienced at reviewing examsin that particular area (e.g., a user that rarely reviews brain MRIs).Thus, importance scores may more closely approximate preferences andknowledge of experts in a particular area (or some other group ofindividuals that are designated to have a higher weighting, such asindividuals within a particular radiology group of a user) without beingskewed by non-experts (or users outside of the particular radiologygroup of the user). Additionally, other aspects of the user's behaviorwith reference to images of respective exam series (including thosediscussed in the following paragraph) may be included as factors in animportance scoring algorithm.

A number of aspects of the user's behavior may be used to determinewhich series are important, and this information may be correlated tovarious clinical indications (and/or other characteristics of an exam).For example, one or more of the following may be used to rank variousseries in terms of importance:

-   -   Order in which various series are viewed by the user    -   Frequency that images for a series type are used for        measurements    -   Frequency that images for a series type are selected by the user        for various operations, for example,        -   Selected as a “key image”        -   Selected for inclusion in a montage of images selected to            summarize the results of the exam        -   Selected for inclusion in a report.

Once the series importance of various series of an exam is determined,the information could be used to increase efficiency of a viewing userin a number of ways. For example, the series importance data (e.g.,series importance scores calculated based on one or more of thecharacteristics listed above) may be used to direct the user's attentionfirst to a series having a highest series importance. Depending on theembodiment, one or more of the following could be used to communicatethe series importance of series to the user:

-   -   Importance could be used to determine the order series are        presented to a user.    -   Importance could be used to determine an order in which series        are transmitted to a user, such as from an imaging center to a        radiologist's viewing device.    -   Particularly important series could be pointed out to users by        highlighting them on a list or visually distinguishing them on a        computer display.    -   Series that are ranked low might be candidates for series that        might be eliminated from imaging protocols, reducing scan time,        or might be placed last in a hanging protocol used by particular        users.

In some embodiments, the series importance data may be used as part of acognitive augmentation application wherein such series importance dataprovides user feedback (e.g., akin to user voting) by one or more useras to the importance, sensitivity and/or relevance of various series invarious clinical indications.

FIG. 5b is a flowchart illustrating one embodiment of a method formonitoring user behavior to collect information related to seriesimportance.

Beginning in block 550, the computing devices determines the user'sidentify so the behavior monitored can be associated with the user. Insome embodiments, interaction data is acquired anonymously or associatedwith a user group rather than an individual user.

In block 552, clinical information associated with the exam may beacquired, such as the clinical indication for the exam. In someembodiments, other clinical information may be utilized, such as theclinical indication, patient's past medical history, risk factors, etc.In other embodiments, other information, such as exam type, may beutilized instead of or in addition to clinical information.Additionally, information regarding the user and/or the user's viewingenvironment (e.g., the type of device the user is viewing the images on)may be acquired.

In block 554, interaction data based on the user's behavior as the userviews the medical imaging exam is recorded. For example, the interactiondata may monitor and include indications of images that the user selectsas “important.” In different embodiments, an image series may be markedas “important” if the user does one or more of the following:

-   -   Selects one or more images of the series to be flagged as a “key        image,” which may indicate that the image series is clinically        important, for example using information recorded in DICOM data.    -   Selects one or more images of the image series to be included in        a “montage” of images that is stored with the exam for the        purpose of communicating relevant findings to other uses that        may view the exam.    -   Performs an operation on one or more images of the image series,        e.g. makes a measurement and/or processes one or more images,        for example using multiplanar reformatting, 3D volume rendering,        and/or Computer Aided Diagnosis software.    -   Manually marks one or more images of the series or the entire        series as one that should be considered important for purposes        of determining series importance.

Depending on the embodiment, a threshold quantity of images of a seriesthat are required to meet one or more of the criteria above may be set,such as based on user, system, site, or default software preferences.For example, one embodiment may require only one image of a series to beflagged as a key image for the series to be marked as important (e.g., auser may set a threshold to one), while another embodiment may requiretwo or more images of a series to be flagged as key images for theseries to be marked as important (e.g., a user may set a threshold totwo).

Moving to block 556, the interaction data, such as the informationdiscussed with reference to block 554, is analyzed to determine thefrequency that images of a series type is tagged as important, forexample, where the number of images tagged within each series weightsthe importance of the series.

In block 558, the interaction data, including the determined seriesimportance data, is stored for use in customizing the user's experiencewith similar exams in the future. In one embodiment, series importancedata may be generated based on the user's viewing behavior for all examsof a certain type, such as a brain MRI. In another embodiment, seriesimportance data may be generated for exams of a certain type coupledwith clinical information, such as a brain MRI performed to evaluate forpossible “acute infarction”. In another embodiment, series importancedata may be created by combining information from multiple users. Thus,in some embodiments blocks 556 and 558 are performed periodically,rather than each time new interaction data is acquired.

In some embodiments, importance of individual images may be tracked inaddition to, or as an alternative to, tracking importance of series ofimages. For example, a particular image of a brain MRI may have a highimportance score in view of the user marking images of that particularanatomy as important in multiple previous exams (relative to a frequencyof the user marking images of other anatomy as important in the samemultiple previous exams). Thus, in one embodiment a computing system maydetermine an order of display of images of a particular series (or ofmultiple series) based on relative image importance data. Similarly, insome embodiments importance of sections of images within image seriesmay be tracked. For example, a brain MRI image series may have multipledifferent sections (e.g., five sections) each comprising multipleimages. In one embodiment, importance scores may be generated for eachof the different sections in order to allow later displays of similarexam series (e.g., from exams of the same type and/or having the sameclinical indication and/or other clinical information) to be optimizedby ordering display of exam sections based on the relative sectionimportance scores. In embodiments where importance scores for individualimages and/or sections of image series are tracked, the system mayinclude registration algorithms that match the anatomy of images (orsections of images within a series) for compilation of importance scoresfor particular images or image sections and for display of appropriatecorresponding anatomy based on stored importance scores.

FIG. 6 illustrates exemplary orderings of series based on seriesimportance for the respective exam indication. In particular, FIG. 6illustrates a “Follow-up Tumor” series importance order 610 indicatingthat the Axial T1+C series was found to be the most “important” seriesbased on user interaction data, followed by the “Axial T1” series, etc.As noted above, depending on the embodiment, series importance may bebased on interaction data for a particular user and/or a group of users.

In this example, the “Multiple Sclerosis” series importance order 620indicates that the “Sagittal FLAIR” series was found to be most“important”, followed by the “Axial FLAIR” series, etc. In this example,the “Acute Stroke” series importance order 630 indicates that the “AxialDiffusion” series was found to be most “important”, followed by the“Axial FLAIR” series, etc.

Because series importance may be customized based on the particularuser(s) interaction data that is used in developing the seriesimportance, different users may have different series importance for thesame exam type and clinical indication. Furthermore, series importancemay change over time as additional user interaction data is obtained andused in determining series importance for a particular user.

FIG. 7 illustrates a computing device as images are selected anddisplayed on a computing device. In particular, view 710 illustrates amobile computing device displaying an image from a brain MRI. In thisexample, the device is displaying an image from the first seriesacquired.

In various embodiments, different methods may be used to allow the userto select other series for display as well as select different imageswithin the series to display. For example, as shown in view 720, a dropdown list may be used to select a series to display. The example dropdown list shown has “Sagittal T1” selected. View 720 illustrates a listof series available for display and allows the user to select anotherseries, for example the “Axial T2” series, which would result in thedisplay of that series.

View 730 illustrates the display after the user has selected the axialT2 series for display, such as by using the drop-down list illustratedin 720. In this embodiment, if the user were viewing an exam for “AcuteStroke” and desired, for example, to routinely view the Axial Diffusionseries first, the system would be inefficient because the user wouldneed to perform the following steps:

-   -   Select the drop down menu to display the list of the various        series    -   Find the Axial Diffusion series within the list. In the example        illustrated the user would need to scroll the list to find that        entry as it is below the entries listed.    -   Select “Axial Diffusion” from the list.

In other embodiments, buttons may be utilized rather than the drop downmenu. In the example shown, the buttons labeled “>>” and “<<” may beused to display the next or prior series, respectively. The buttonslabeled “>” and “<” may be used to display the next or prior imagewithin a series. In other embodiments, touch actions, such as left andright finger swipes to advance to the adjacent series, may be utilizedto change series and images within series. However, use of the buttonsand/or touch actions to navigate between image series that are stored inthe original acquisition order or in alphabetical order introducessimilar inefficiencies as use of the drop-down list. For example, if animage series that a user routinely prefers to view first is positionednear the end of a series list, the user may have to push the seriesadvanced button multiple times, each time checking to see which seriesis display, in order to get to the desired series.

Example Applications of Series View Order and Series Importance

More efficient methods for allowing the user to quickly and easily viewimage series of most importance are discussed below.

FIG. 8 illustrates a computing device as images are selected anddisplayed on the computing device. In particular, view 820 of FIG. 8illustrates the mobile computing device displaying an image from a brainMRI. In this embodiment, the series are ordered based on the user'stypical or desired series view order. For example, the user's seriesview order may be determined based on stored interaction data associatedwith user, and also associated with the particular exam type and/orclinical indication. Thus, the order in which the user previously viewedseries for the current clinical indication may be used in order todecrease navigation required by the user to view series of the newlyselected exam in that same order.

In some embodiments, the series may be arranged in order of importance,based on the systems and methods described herein. For example, thedevice 820 may display an image from the image series having the highestseries importance (rather than from the first series acquired as in FIG.7). As discussed herein, the user's profile (and possibly interactiondata of other users) may be used to determine series importance forvarious series associated with a particular clinical indication, in thiscase “Acute Stroke.” Thus, the interaction data can be used to generatea custom order for presentation of image series that is different thanthe order acquired. For example, the series order for a brain MRIperformed for “Acute Stroke” might be in the order shown in view 630 inFIG. 6. Therefore the first series displayed would be the “AxialDiffusion” series, as illustrated in view 820.

Using certain systems and methods described herein, view 830 illustratesthe list of series that might be displayed on a handheld computingdevice if the user touched the drop down list. Note the order of theseries listed is the same as shown in the example of view 630 of FIG. 6,which is a custom series ordering based on series importance data.

In some embodiments, the series order presented to a user may bedetermined based on a combination of series view order for theparticular user as well as series importance data for the user (andpossibly other users). Depending on the embodiment, the user may be ableto customize the relative importance of having the series ordered basedon series view order as opposed to series importance. For example, afirst user may indicate that the series order for a particular examtype, with or without associated clinical information, should be basedon primarily (e.g., 80%) on the users series view order, with someconsideration (e.g., 20%) for series importance data from a group ofspecialists in the field. Likewise, a second user may indicate that theseries order for the same exam type should be based on primarily (e.g.,75%) series importance data for the user, with some small consideration(e.g., 25%) for series view order for the user. Thus, the user isprovided with various levels of customization to allow the computingdevice to intelligently determine a most appropriate ordering of seriesof an exam.

FIG. 9 illustrates additional views of the image series discussed withreference to FIGS. 7 and 8, again with the series ordered according toseries view order and/or series importance, as discussed with referenceto FIG. 8.

View 910 illustrates an image from the first series in the custom orderillustrated in series importance order 630 (FIG. 6), the Axial Diffusionseries. When the user is done viewing the images in that series, he mayadvance to the next series, for example by pressing a button, e.g., theone labeled “>>”, or by a touch gesture, as illustrated in FIG. 10. View920 illustrates the display of an image from the next series, the AxialFLAIR series.

This workflow allows the user to advance through the series, from mostto least “important” without needing to display the list of series andmanually select from the list. In another embodiment, the series may belisted in order of the user's preferred series view order as describedherein.

View 930 illustrates the list of the various series, for example inresponse to the user selecting the drop down menu, ordered, for example,based on series view order or series importance, as described herein.

FIG. 10 illustrates an embodiment where the user may change series andimages using touch gestures. View 1010 illustrates a device thatincludes a touch screen display. View 1020 illustrates gestures that auser might use to change the image displayed within a series. Forexample, touching the screen and moving the finger up might advance tothe next image within a series, while touching the screen and moving thefinger down might display the prior image in the series. View 1030illustrates gestures that a user might use to change the seriesdisplayed within the exam. For example, touching the screen and movingthe finger left might advance to the prior series, while touching thescreen and moving the finger right might display the next series.

FIG. 11 is a flowchart illustrating one embodiment of a method ofpre-loading series based on a custom series order, such as based onseries view order of the user and/or series importance for variousseries. In some scenarios, real-time communication of images to thecomputing device may be too slow to support the performance desired bythe user, for example when image communication occurs over the internetor via cellular data networks. Preloading images into the memory of adisplay computing device or into relatively fast local storage mayreduce and/or overcome this problem. This process may be optimized bypreloading images in the order they are likely to be needed for displayby the user, referred to here as the “Series Loading Priority.” Based onsystems and methods described herein, the series loading priority may bebased on series view order, series importance, and/or set explicitly,for example as a user, group, or site preference.

Beginning in block 1110, the computing device determines the user and/oruser group specific for which the exam is to be transferred. Dependingon the embodiment, the preloading order may be determined by the actualdisplay computing device (e.g., a radiologist's tablet) or by a networkdevice, such as a PACS server or electronic medical records system, forexample. Thus, discussion of processes performed by a computing devicemay refer to one or both of the client (e.g., the doctors computingdevice) or server (e.g., the image server).

In block 1115, clinical information associated with the exam may beacquired in embodiments where that information is used to determine theseries loading priority, such as when series view order and/or seriesimportance are associated with specific clinical information (e.g.,clinical indication and/or exam type).

Next, in block 1120 the list of series associated with the exam isretrieved and in block 1125 the series loading priority is determinedand/or retrieved, e.g., from the user profile data 160 of FIG. 1. Invarious embodiments the series loading priority may be based oninformation collected on the individual user or group of users. In otherembodiments the series loading priority may be predefined for the user,user group, site and/or system preference.

In block 1130 the series are transferred to the computing device in anorder based on the series loading priority, for example from a server tothe user's computing device or LAN.

In another embodiment, the series loading priority is utilized toprioritize the order of a function other than transfer of series. Forexample, the series loading priority might be utilized to orderprocessing of images prior to display, for example image decompressionor creation of MPR (multiplanar reconstruction) images based on a user,user group, site, or system protocol for automatic generation of MPR or3D volumetric rendered images.

FIG. 12 is a flowchart illustrating one embodiment of the method ofpresenting series of an exam based on determined series importance.

Beginning in block 1210, the computing device determines the user and/oruser group specific to which the image series will be presented. Asdiscussed above, the order of presenting image series of the exam may becustomized based on preferences of the particular user and/or groups towhich the user is a member.

In block 1215, clinical information associated with the exam may beacquired in embodiments where that information is used to determine theseries importance.

In block 1220, the list of series associated with the exam is retrieved.

In block 1225, the series importance data is retrieved, for example fromthe User Profile Data 160 of FIG. 1. In various embodiments the seriesimportance may be based on interaction data collected on the individualuser or group of users. In other embodiments the series importance maybe predefined for a user, user group, site and/or system preference.

In block 1230, the series are displayed by the computing device in anorder or image configuration based on the series importance.

FIG. 13 illustrates an example screen from a computing device configuredto display images from multiple image series concurrently. In theexample illustrated, six image frames are shown, each displaying aseries from a medical imaging exam. In this example, the series are eachfrom a brain MRI exam and the series types are indicated in the imageframes, e.g., Sagittal T1, Axial T1, etc. Any number of image frames maybe utilized and they may be displayed on one or more display devices.The operations described here could occur on any computing device, suchas a PC, workstation, tablet computer, handheld device, etc. Within eachframe the user may display other images within the associated series,for example by performing certain operations with a computer mouse suchas holding down the left mouse button and moving the mouse up or down,by clicking on a button displayed on the computer screen (not shown), bypressing a key on a keyboard, via a gesture on a touch screen, etc.

The user is free to view the images within the frames in any orderdesired and change the series displayed in each frame, for example byrearranging the series displayed on the screen or displaying series thatare not displayed, for example if there are more series than imageframes.

The initial arrangement of series on the display may be determined by ahanging protocol, for example specific to the user, and the user mayprefer the example shown in view 1310 for brain MRI exams, regardless ofthe clinical indication. However, based on the clinical informationassociated with the exam, the user may choose to review the variousseries in a different order depending on the clinical informationassociated with the exam.

When a radiologist or other user chooses a medical imaging exam todisplay on a computing device it may require a significant amount oftime for the images associated with the exam to be transferred to thecomputing device, particularly if a slow network, internet, or cellulardata network is utilized. The time required to transfer the informationmay result in user frustration and decreased efficiency as the userwaits for the information to be transferred.

Using systems and methods described herein, the information that theuser is likely to display first may be prioritized so that it isavailable first. For example, for a patient with a clinical history of“Acute Infarct,” the user may prefer to view images in the AxialDiffusion series first because the user believes that the AxialDiffusion series is the most sensitive for detection of acuteinfarction. Based on systems and methods described herein, the systemmay be configured to load the Axial Diffusion series first so that itwould be available for the particular user to view immediately. Incontrast, when an exam is selected with a different clinical indication,such as “Follow-up Multiple Sclerosis,” the user may prefer to view the“Sagittal FLAIR” images first, so that series would be given the highestpriority.

It is noted that the series loading priority may be independent of thearrangement of the series on the display device, for example asdetermined by hanging protocols.

In other embodiments, the series view order and/or series importancedata may be utilized to automatically control the arrangement of theseries on the display device, such as by modifying or generating hangingprotocols. This may provide cognitive support, for example to non-expertusers. For example, the series importance data from a group ofradiologists or other experts may be determined and used to customizethe arrangement of series on the display device, such as by generating acustomized hanging protocol for the particular clinical indication. Forexample, the series orders shown in FIG. 6 might represent the seriesimportance information from a group of expert readers for brain MRIexams performed for three different clinical indications. View 1320illustrates an example from an embodiment where the series are arrangedon the display device according to their importance. In this example, abrain MRI was performed with the clinical indication of “Acute Stroke.”As a result of the application of certain systems and methods describedherein, the series are arranged in order of importance, from left toright and top to bottom. Specifically, the series order is AxialDiffusion, Axial FLAIR, Axial T2, Axial T1, Sagittal T1 and Coronal GRE,corresponding to the series order shown in view 630 of FIG. 6.

In some embodiments, the series loading priority (discussed withreference to FIG. 11) and the series importance data are both utilizedin order to optimize the availability of the most important image seriesand arrange the image series in an order that highlights those of mostimportance.

In another embodiment, the arrangement of series on a display device,for example as discussed with reference to view 1320 above, is based onseries view order rather than (or in combination with) seriesimportance.

FIG. 14 illustrates an arrangement of various image series, wherein sixof the image series may be displayed on a display device (or multipledisplay devices) concurrently. In particular, view 1401 of FIG. 14illustrates image series that are similar to those displayed in view1320 of FIG. 13. However, as is illustrated in FIG. 14, the number ofseries exceeds the number of image frames displayed on the displaydevice leaving four image series not displayed. FIG. 15 illustrates atablet device displaying the first six image series of the exam.

In the example show in FIG. 14, a contrast enhanced brain MRI wasperformed with a clinical history of “Possible Brain Metastases.”Selection 1401 indicates a portion of the series that are capable ofbeing displayed currently on the display device, e.g., the exampledisplay device can display six series concurrently. In this embodiment,the image series are ordered by importance, arranged from top to bottomfollowed by left to right. In this example the order of seriesimportance is Axial T1, Coronal T1+C, Axial T1+C, Sagittal T1+C, AxialFLAIR, Sagittal T1, Axial T2, Axial Diffusion, etc.

Selection 1410, displayed with a dashed line, outlines the first sixseries that might be first displayed automatically on the display deviceillustrated in FIG. 15, as shown. By interacting with the computingdevice, for example by using a left finger swipe on the touchscreen of adevice, as illustrated in view 1030 of FIG. 10, the series displayed onthe device could be changed. For example, a left swipe might display theseries selection 1420, bringing in the next two series, in order ofimportance. Left and right swipes could conceptually move box 1420 leftor right, displaying series of greater and lesser importance.

The systems and methods described herein may increase the accuracy ofreaders by presenting first the series that are most likely to beimportant in various clinical situations, directing the reader'sattention to those series. In another embodiment, the series arearranged according to a series view order rather than (or in additionto) series importance.

Other

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art.

All of the methods and processes described above may be embodied in, andpartially or fully automated via, software code modules executed by oneor more general purpose computers. For example, the methods describedherein may be performed by an Information Display Computing Deviceand/or any other suitable computing device. The methods may be executedon the computing devices in response to execution of softwareinstructions or other executable code read from a tangible computerreadable medium. A tangible computer readable medium is a data storagedevice that can store data that is readable by a computer system.Examples of computer readable mediums include read-only memory,random-access memory, other volatile or non-volatile memory devices,CD-ROMs, magnetic tape, flash drives, and optical data storage devices.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure. The foregoing description details certainembodiments of the invention. It will be appreciated, however, that nomatter how detailed the foregoing appears in text, the invention can bepracticed in many ways. As is also stated above, it should be noted thatthe use of particular terminology when describing certain features oraspects of the invention should not be taken to imply that theterminology is being re-defined herein to be restricted to including anyspecific characteristics of the features or aspects of the inventionwith which that terminology is associated. The scope of the inventionshould therefore be construed in accordance with the appended claims andany equivalents thereof.

What is claimed is:
 1. A method of pre-loading a plurality of imageseries associated with a medical exam into a memory of a displaycomputing device, the method comprising: determining, by one or morehardware computer processors executing computer-executable instructionsstored on one or more non-transitory computer-readable mediums, a useror a user group for which the medical exam is to be transferred;acquiring, by the one or more hardware computer processors, clinicalinformation associated with the medical exam; retrieving, by the one ormore hardware computer processors, a list of the plurality of imageseries associated with the medical exam; determining, by the one or morehardware computer processors, a series loading priority of the pluralityof image series associated with the medical exam based on the clinicalinformation associated with the medical exam; and transferring at leasta subset of the plurality of images series to the memory of the displaycomputing device in an order based on the series loading priority. 2.The method of claim 1, wherein determining the series loading priorityof the plurality of image series associated with the medical examincludes determining the series loading priority based on the clinicalinformation associated with the medical exam and profile data associatedwith the user or the user group.
 3. The method of claim 1, whereindetermining the series loading priority of the plurality of image seriesassociated with the medical exam includes determining the series loadingpriority based on the clinical information associated with the medicalexam and a predefined priority for the user or the user group.
 4. Themethod of claim 1, wherein determining the series loading priority ofthe plurality of image series associated with the medical exam includesdetermining the series loading priority based on the clinicalinformation associated with the medical exam and a predefined priorityfor a site or a system associated with the user or the user group. 5.The method of claim 1, wherein determining the series loading priorityof the plurality of image series associated with the medical examincludes determining the series loading priority based on the clinicalinformation associated with the medical exam and information collectedon the user or the user group.
 6. The method of claim 5, whereindetermining the series loading priority based on the informationcollected on the user or user group includes determining interactiondata for the user or the user group, wherein the interaction dataindicates for at least one of the one or more previous medical examsassociated with the determined clinical information, indications offrequencies of images of each respective series type of the previousmedical exam being marked as important by the user or the user group;and determining the series loading priority based on the interactiondata.
 7. The method of claim 6, wherein an image marked as important isindicated by at least one of: the image being added to a montage; theimage being marked as a key image; the image being selected for displayin a particular order with respect to other images; a measurement beingperformed on the image; or the image being selected for inclusion in areport.
 8. The method of claim 5, wherein determining the series loadingpriority based on the information collected on the user or user groupincludes determining interaction data including, for one or moreprevious medical exams associated with the clinical information,indications of frequencies of images of respective series types of theprevious medical exams being marked as important by a user designated asan expert with respect to medical exams associated with the clinicalinformation, wherein each respective series type indicates at least oneof an imaging orientation, imaging modality, or an imaging plane;determining, based on the interaction data and by the one or morehardware computing processors, a first series type of the respectiveseries types having a highest frequency of images previously marked asimportant; determining, based on the interaction data and by the one ormore hardware computing processors, a second series type of therespective series types having a second highest frequency of imagespreviously marked as important; and determining, based on theinteraction data and by the one or more hardware computing processors,the series priority loading priority of the respective series typeshaving the second highest frequency of images previously marked asimportant.
 9. The method of claim 1, wherein transferring at least asubset of the plurality of images series to the memory of the displaycomputing device in an order based on the series loading priorityincludes transmitting a first image series included in the plurality ofimages series before transmitting a second image series included in theplurality of images series when the first image series is ordered beforethe second image series within the series priority loading priority. 10.The method of claim 1, wherein acquiring the clinical informationassociated with the medical exam includes acquiring one selected from agroup consisting of a clinical indication and an exam type.
 11. Themethod of claim 1, wherein transferring at least a subset of theplurality of images series to the memory of the display computing deviceincludes transferring at least the subset of the plurality of imagesseries from a server to the display computing device or a local areanetwork.
 12. The method of claim 1, further comprising processing, bythe one or more hardware computer processors, at least the subset of theplurality of images series in the order based on the series loadingpriority.
 13. The method of claim 12, wherein processing at least thesubset of the plurality of image series in the order based on the seriesloading priority includes processing at least the subset of theplurality of images series in the order based on the series loadingpriority prior to displaying at least one of the plurality of imageseries.
 14. The method of claim 12, wherein processing at least thesubset of the plurality of images series in the order based on theseries loading priority includes performing one selected from a groupconsisting of image decompression, creation of multiplanarreconstruction images, and three-dimensional volumetric rendering in theorder based on the series loading priority.
 15. The method of claim 12,wherein processing at least the subset of the plurality of images seriesin the order based on the series loading priority includes processing,with computer aided diagnostics and by the one or more hardware computerprocessors, at least the subset of the plurality of image series in theorder based on the series loading priority.
 16. A computing system forpre-loading a plurality of image series associated with a medical examinto a memory of a display computing device, the system comprising: oneor more hardware computer processors configured to execute softwareinstructions to at least: determine a user or user group for which themedical exam is to be transferred; acquire clinical informationassociated with the medical exam; retrieve a list of the plurality ofimage series associated with the medical exam; determining, by the oneor more hardware computer processors, a series loading priority of theplurality of image series associated with the medical exam based on theclinical information associated with the medical exam; and transfer atleast a subset of the plurality of images series to the memory of thedisplay computing device in an order based on the series loadingpriority.
 17. The computing system of claim 16, wherein the one or morehardware processors are included in the display computing device. 18.The computing system of claim 16, wherein the one or more hardwareprocessors are included in a picture archiving communication system oran electronic medical records system.
 19. The computing system of claim16, wherein the one or more hardware processors are configured todetermine the series loading priority of the plurality of image seriesassociated with the medical image by determining interaction data forthe user or the user group, wherein the interaction data indicates forat least one of the one or more previous medical exams associated withthe determined clinical information, indications of frequencies ofimages of each respective series type of the previous medical exam beingmarked as important by the user or the user group; and determining theseries loading priority based on the interaction data.
 20. Anon-transitory computer-readable storage medium storing softwareinstructions that, in response to execution by a computer system havingone or more hardware processors, configure the computer system toperform operations comprising: determining a user or a user group forwhich the medical exam is to be transferred; acquiring clinicalinformation associated with the medical exam; retrieving a list of theplurality of image series associated with the medical exam; determininga series loading priority of the plurality of image series associatedwith the medical exam based on the clinical information associated withthe medical exam and data associated with the user or the user group;and transferring at least a subset of the plurality of images series tothe memory of the display computing device in an order based on theseries loading priority, wherein the data associated with the user orthe user group includes one selected from a group consisting of profiledata associated with the user or the user group, a predefined priorityfor the user or the user group, a predefined priority for a site or asystem associated with the user or the user group, and interaction datafor the user or the user group, wherein the interaction data indicatesfor at least one of the one or more previous medical exams associatedwith the determined clinical information, indications of frequencies ofimages of each respective series type of the previous medical exam beingmarked as important by the user or the user group.